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How good are global Newton methods. Part 2 
A. A. Goldstein* 



In the first part of this ms. we made some observations about global Newton methods 
on a simple class of mappings from a real separable Hilbert space H into H. The map 
F was assumed to be everywhere differentiable with a derivative D that was onto and 
satisfied ^||/j|| < ||Z)(x)/i|| < A||/j||, for all h G and some ^ > 0. Relying on theorems of 
Nemerovsky and Yudin(1977), we showed that no formulation of a global Newton method 
could achieve better than linear convergence at a certain specified rate for every member 
of the class. Assuming that D was uniformly Lipschitz continuous with constant L it 
was easy to utilize the Kantorovich inequalities (1948). Points satisfying the Kantorovich 
inequalities are guaranteed to be a satisfactory initial point for Newton’s method. By 
satisfactory, we mean that the sequence converges to a root at a quadratic rate, as will 
be seen below. Following a similar terminology of Smale(1986), such points will be called 
approximate roots. 

We used the Kantorovich inequalities to formulate a coarse version of Smale’s global 
Newton method to obtain the complexity of the algorithm in this setting. The Kan- 
torovich condition is : if xq satisfies ||T)“^(xo)|| ||i?~^(xo)F'(xo)||X' = 0t]L < .5 

then xq is an approximate root. Here L is the Lipschitz constant for D on the ball 
B = {x G : ||x — xo|| < 2||T) ^(xo)B(xo)|l}. We found that starting at xq we could 
achieve a point satisfying the Kantorovich inequalities in less than: 

11.46 [||F(xo)||L///^ ]7n (1.443 /n 8Q) steps 

where: L is the global Lipschitz constant, and Q = X/p the condition number. It is 
noteworthy that this algorithm is insensitive to the condition number. When the word 
‘steps’ appears in this paper, as above, we mean the preceding formula to be rounded up 
to the nearest integer. 

Some numerical experiments however were disappointing. The poor performance stemmed 
from our reliance on the Kantorovich inequality. This inequality is over conservative when 

* supported by grant NPS LMC-M4E1 
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the dimension of H exceeds 1. In a test example even though we estimated L along only 
one ray (in the direction of the Newton step) and thus determined a L that was a lower 
bound, we found approximate roots where the Kantorovich constant /3 t] L was several 
thousand instead of < 1/2. (See definition 1.5 below). For this reason we undertook to 
revise the Kantorovich inequalities, hoping to obtain better sufficient conditions. We shall 
do this by avoiding the Schwarz inequality at a key spot. In general we were guided by 
Kantorovich’s original proof. A parameter K replaces L on the ball B. Estimates are given 
for the decrease in the norm of the vector valued function. This leads to the definition of a 
an e approximate root. We believe this concept will be useful in the further development of 
global Newton algorithms. The estimates we give for the convergence rates can probably 
improved by a one dimensional analysis, as was done by Gragg and Tapia(1974) for the 
Kantorovich Theorem.. 

One consequence of the revised proof is that the derivative operator need not be continuous, 
although Lipschitz continuity is convenient in a neighborhood of each root. As an example 
we consider the problem of solving linear inequalities. (Phase 1 linear programming) By 
the use of Lagrange multipliers the method presented here can be extended to linear pro- 
gramming and other types of programs. We shall however only consider linear inequalities 
in this ms. The idea of using Newton’s method for linear programming may be found in 
Smale(1986). Our technique will be to use a penalty function approach. In this setting 
we have a twice differential convex function to minimize, with jump discontinuities in the 
second derivative. For the code we refurbished an old global Newton method of our own. 
This was easy because it was already up and running. We tried the method on an example 
due to C. Blair(in Kortanek and Shi 1987) that is a version of the famous Klee-Minty 
example. This problem was suggested to us by Ken Kortanek, and we are thankful for his 
kind help. While Ken was visiting, he ran his scaling (modified Karmarker) algorithm on 
my computer for a benchmark in the 10 variable case. The robust ‘Superlindo’ by Linus 
Schrage was also used as a benchmark for these problems. For 8, 10, and 12 variables 
Superlindo took 35, 311, and 2236 iterations respectively. Our algorithm hardly changed 
in the number of steps versus dimension. Ken Kortanek remarked he noticed the same 
phenomena with their algorithm. In the appendix appears a list of 15 consecutive runs 
for the case n=12 with the components of the initial point being fed by a random number 
generator with values between -1000 and 1000. The number of steps until termination ran 
between 6 Newton steps plus 2 gradient steps, and 12 Newton steps plus 3 gradient steps. 
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The average was 8 Newton steps and 19/15 gradient steps. Full double precision accuracy 
was achieved. The Kortanek-Shi algorithm in the 10 variable case was almost as fast as 
ours for this case, but it achieved only 3 significant figure accuracy. 

These spectacular results were unexpected. Unlike the Smale global Newton method, 
which we as yet have not implemented, the method we used is sensitive to the condition 
number of the system. See 3.0 below. 

There remains to test the algorithm on a general mix of problems to see if it is worth 
adding to the armamentarium of linear programming. Thus the question of whether we 
have a viable method is not settled at this time. 
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In what follows, the Kantorovich inequality will be sharpened. Let f be Frechet differen- 
tiable mapping between real Banach spaces E and F. Let f\x) denote the derivative at x 
and f'_i{x) its inverse. The Kantorovich inequality states that if a:o is given such that /' is 
Lipschitz continuous with constant K on the ball {x 6 E' : ||x — xo|| < 2 ||/L 2 ( 2 ^o)/(a;o)l| = 
2 Tjo], if ||/-i(3;o)|| = 00 and if r]o0oK < 1/2, then xq is an approximate root. This 
means that the sequence generated by Newton’s method will converge quadratically to a 
root at least as fast as the rate estimated by the theorem below. 



THEOREM 1.0 Let f be a map between real Banach spaces E and F. Assume 
f is Frechet differentiable on an open subset E' of E. Let xq € E' be given such that 
(/'(xo))“^ = fLiixo) exists. Set 

Vo = ||/ii(3:o)/(a:o)|| 



S = { X G E : ||x — xo II < 1.68 Vo }■ Assume that S C E' 

Set 



00 = ||/ii(a;o)|| , and let 

5' = {xG5:||/L2(x)|| <2.3^o}. 



Finally set: 



K = 



sup 



- nm 

WfL,{x)\\ ||/L2(x)/(x)|| 



X € s' , ^ = X -|- tf'_i (x)/(x) and t € (0, 1) 



If vo0oK = ho < 1/3, then xq is an approximate root. 

PROOF By hypothesis Xi is well defined. Let Ei(xo,xi) = Hi = f'^i{xo)f'{xi) = 
I — f'-i{xo){f'{xo) — /'(xi)). Hi maps E into itself. Our hypotheses imply that 
||/Li(a:o)(/^(a;o) — /^(a:i))l| < ho < 1/3, whence Hi has an inverse. We have the estimate 
|l(/f2)-i|| = (1 - ho)-^ < 3/2. Also llEill < 4/3. Observe that f'{xo)Hi = f'{xi) and 
{Hi)~^ f'_i{xo) = f'_i{xi). Thus f'_i{xi) exists and X 2 is well defined. 



Let 01 = ||/Li(xi)|| and vi = Iki ~ X 2 \\. Thus 0i = ||/Li(xi)|| < 1.5 ||/Li(xo)l|. 
Let El be defined by the formula Ei(x) = x — /Li(xo)/(x). Since Ei(xo) = xi and 

Ei(xi) = xi - /Li(xo)/(xi), 

f'-i{xo)f{xi) = Fi(xo)-Fi{xi). 
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REMARK 1.1 Assume the hypotheses of the theorem with suitable changes in S and S' . 
Take ho < 1/2. Then xo is an approximate root. 

REMARK 1.2 The theorem does not require the continuity of /'. Let f(x) = 4x, if 
oo < X < 1 and f(x) = 6x -2, if x > 1. Then every point is an approximate root for 
f. For example let xq = 100. Then ho = 1/3, xi = 1/3, h\ = 0, and X 2 = 0. 

REMARK 1.3 Let S, S' 2 md K be defined as above. Assume that f is Lipschitz continuous 
with constant L on S. Let hk = sup{||/Lj(xfc)(/'(xo) — /'(Oil • ^ ^ S’ and ^ = Xk + 
G (0, 1)}. Assume the Kantorovich inequality holds. Then hk < ^kVkL < 
1/3 . Whence hk/PkVk = K < L . We have replaced L by K. Another hypothesis to ensure 
the boundedness of K (which we’ve assumed outright) is to require that /' be bounded on 
S' and, for any root x* of f there is a neighborhood N(x*) e such that f is Lipschitz 
continuous on N(x*)D5'. 

REMARK 1.4 The Kantorovich type theorems are important for the insight they furnish, 
but in general are not helpful in concrete instances. For example, the above theorem and 
the original Kantorovich theorem are dependent upon the knowledge of the constant K or 
L, but the determination of upper bounds for these quantities is not trivial. Moreover the 
conditions given by these inequalities is sufficient but not in general necessary for a point 
to be an approximate root. Thus the determination of approximate roots is usually not 
practical with these theorems. Using the rough estimates furnished by local application of 
the modified Kantorovich inequality we propose proceeding by trial. 

DEFINITION 1.5 Given e > 0, let n be the smallest integer such that 




A point Xo is an e- approximate root for the mapping f if the Newton sequence {x;} 
starting at xq is well defined for i=l,2,..., n, and 

ll/(^n)|| ^ ^ 

ll/(a:o)|| 

For example if e = 10“^® then n = 7. This test follows from the estimates I and II of the 
above theorem. 
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LEMMA 2.0 Let A be a positive definite matrix. Let // be its least eigenvalue and A its 
greatest. Let Q = A///. Then: 



[Ax, x] 2 ^ Q 
ll^^ll ll^ll ~ {Q A- 1) 

The bound is the best possible. 

PROOF. Assume A has been diagonalized, and let 

[Ax,x]jn = x] + C2xl + ,...,Cnx'^j^. 

We seek to minimize f{x) = (xj + C2X^+, ...,C4X^)/(xj + <^x\ =,..., 2, subject to 

||a:|| = 1. The reason for restricting to polynomials of degree 4 in xf will become apparent 
in what follows. 

Set Xj = 1 — X2 — X3 — X4. Using this equation to eliminate x\ we get: 

r,s ^ 1 + (C2 - l)x^ + (C3 - l)xf + (C4 - l)x^ 

(1 + (c] - l)x 2 + {cj - l)xl + (cl - l)xl)2 

Let P denote the numerator of the above fraction and Q 2 the denominator. Let x = 
(xl, xl, X4) and 

A = (C2 - 1, C3 - 1, C4 - 1) 

and 

B = (cl -I, cl- 1 , cl- 1 ) 

The components of the equation V/(x) = 0 may be written as : 

Xz(c2 - 1)[2B - (C2 + 1)A, x] = X 2 (Cc - 1)(C2 ~ 1) (a) 

Xz(c3 - 1 )[ 2 B — (C3 + l)A, x] = X3(C3 - 1)(C3 - 1 ) (b) 

X4(c4 — 1)[25 — (c4 + 1)A, x] = X4(c4 — 1)^ (c) 

If we assume distinct eigenvalues and that X2, X3 and X4 are non-zero we may cancel 
appropriately and obtain from (a), (b), and (c), a system of 3 linear equations of rank 2, 
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since the rows of the system are linear combinations of A and B. Thus one unknown say X 2 
or X3 (but not X4 ) may be set to 0. Similarly if we had n variables only one variable among 
X2 and x„-i would be non-zero. If we set X2 = 0, then the following system obtains: 

(c| - l)x| -f- (C4 - 1)(C4 - C 3 )x\ = C3 - 1 

{c\ - l)xl -t- {cl - l)xl = C4 - 1 

One finds that 

2 ^ (4 - 1)(C4 -C3) 

-(C3 - 1)(C3 -1- 1)(C4 - 1)(C4 - C3) 

Since x\ is negative, it is inadmissible. Thus x^ must also vanish. Suppose now that X 3 = 
0 but X 2 ^ 0. We again get x\ < 0. Thus for these cases we must set X 2 and X 3 to 0. 

If C2 > 1 and C4 > C2 then the rank of {A,B} = 2 This case is treated just as the case 
above of distinct roots. If all the eigenvalues are equal, the ratio in the statement of the 
lemma is 1, verifying the claim for that case. For all other cases the rank of {A, B} = 1, 
and we may set X 2 and X3 to 0. 



There remains to carry out the minimization. The equation 



(C4 - 1)X4 = C4 - 1 



implies 



and 

2 -^ 

m = 




1 

Q + 1 



Algorithm 3.0 

Let f be a twice differentiable function defined on real euclidean space En- Assume that 
f is convex and that it has bounded level sets. Given an arbitrary point Xg let S denote 
the level set {x € Rn '• f{^) < Assume that the hessian of f at x, which we call 

H(x), has an inverse at xg, denoted by H ^(xg). At the kth step of the algorithm assume 
that H~^{xk) exists. Let 

^ f{x)-f{x-jH-\x)Vf(x)) 

7[if-i(x)V/(x), V/(x)] 
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since the rows of the system are linear combinations of A and B. Thus one unknown say X2 
or 0:3 (but not X4 ) may be set to 0. Similarly if we had n variables only one variable among 
X2 and Xn-i would be non-zero. If we set X2 = 0, then the following system obtains: 

(4 - l)a;l + (C4 - 1)(C4 - C 3 )xl = C 3 -I 

(4 - l)xl + (c^ - 1 )X^ = C4 - 1 

One finds that 

2 ^ (cl -1)(C4 -C3) 

^ -(C3 - 1)(C3 + 1)(C4 - 1)(C4 - C3) 

Since x^ is negative, it is inadmissible. Thus x^ must also vanish. Suppose now that X3 = 
0 but X2 ^ 0. We again get X4 < 0. Thus for these cases we must set X2 and X3 to 0. 

If C2 > 1 and C4 > C2 then the rank of {A,B} = 2 This case is treated just as the case 
above of distinct roots. If all the eigenvalues are equal, the ratio in the statement of the 
lemma is 1, verifying the claim for that case. For all other cases the rank of {A, B} = 1, 
and we may set X2 and X3 to 0. 

There remains to carry out the minimization. The equation 

4 — l)a ?4 = C 4 — 1 implies x\ 

and 

2-^ 

/(X) = 



1 

Q + 1 



Algorithm 3.0 

Let f be a twice differentiable function defined on real euclidean space En- Assume that 
f is convex and that it has bounded level sets. Given an arbitrary point Xq let S denote 
the level set {x € Rn ' f(x) < /(^o)}- Assume that the hessian of f at x, which we call 
H(x), has an inverse at xo, denoted by H~^{xo). At the kth step of the algorithm assume 
that H~^(xk) exists. Let 

/(x)-/(x-TJf-‘(x)V/(x)) 

7|/f-'(x)V/(i), V/(i)l 
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Choose 8 G ( 0 , 1 / 2 ] and choose jk > 0 so that 6 < g{xk,'fk) < 1 — taking 7* = 1, 

if possible. Set Xk+i = Xk - 7* V/(xfc). Let S' = {x^ E S : k = 1 , 2 , 3 , }. 

Assume that the spectrum of H(x) is bounded above on S' by AQ, and below by A. Let 

h = supi^\\H-\x){H{x) - H{ 0 )\\ :xeS' and ^ = x + tH~\x)V /(x), t G [0,1]}. 



(a) Assume that K = 



sup 



rj£_j^ 

\WH-Hx] 



i(x)|| \\H-^(x)Vf(x 
Let N be the least integer exceeding 
/(xo) - minf 



: X E S' and ^ = x + tH ^(x)V/(x), t G [ 0 , 1]| < 00. 



where A = 



<5^A^ 



A (1 + 2/iQ1/2)A:2Q1/2 

Then an approximate root of V/(x) will be achieved in at most N steps. 



(b) Let 



Ai = 



6^(1-<?)^A^ 

{l + 2hQ2 



. Let M be the least integer exceeding: 

/(xq) - minf 
min{A, Ai} 

. Then in at most M steps an approximate root will be achieved, and all subseqent steps 
will be Newton steps. 

PROOF (a) Assume in what follows that x G 5 . The change in f do to the step 
7s(x) = 7if“^(x) V/(x) may be estimated by Taylor’s formula as: 



(x=) A(x) = /(x) - /(x - 7 5 (x)) = 7 



^ _ 7 _ 7 [s(x), (H{() - Jf(x)) s(x)] 



2 2 



[V/(x),s(x)] 



[V/(x),s(x)] 



or 



(**) A(x) = 7^(x,7)[V/(x),s(x)] 

Here ^ lies between x and x - 7 5(x). By the above formula we see that the right hand 
limit of g(x,j) at 7 = 0 is 1. Since S is bounded, for some 7, g(x,j) = 0 . This shows that 
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7 may be chosen as claimed. We see also that {/(x*)} is a decreasing sequence so that 
Xk E S for k=l,2,3, 

We aim now to find a lower bound for A(x). In the equation (*) above, the third term in 
the large brackets is 

' T l|V/(x)|| ||g-‘(x)(g(x) - ffCOII ||.(x)|| 

- 2 (V/(x),/f-i(T)V/(j)] 

Using lemma 2.0 we get the inequalities 

1 - I + |«/i(x)Q 2 > g{x,j) > I -^ah(x)Q 2 . 

Where h(x) = sup {||ff “^(x)(/T(x) - H{()\\ : ^ = x + ts{x) with t € [0, 1]}. and: 

Q + 1 



a = 



2Q ■ 



so that .5 < o < 1. Since g(x, 7) < 1 — 6 we thus obtain the lower bound 

26 



7 > 



1+0 h(x)Qi 



If {||V/(xfc)||} does not converge to 0, an infinite subsequence of it is bounded away from 
0. Since [V/(x),s(x)] > ||V/(x)p/QA, and Jkg{xki7k) is also bounded away from 0, 
f{xk) tends to —00. This contradiction establishes that the sequence {V/(xfc)} converges 
to 0. Moreover every cluster point of the set S' minimizes f, and {f(xk)} converges to 
min f. 

If X does not satisfy our condition for an approximate root then ||if“*(x)|| ||•s(x)|| K > 
1/2. By the lemma [V/(x), s(x)] > ||s(x)|| || V/(x)||. Consequently, ||s(x)|| > 

1/2K ||/f“^(x)||, and ||V/(x)|| ||>s(x)|| > 1/4A”^A^. Thus a lower estimate of (**) is 



A(x) > 



6 ^ 



{l + 2hQ^ 



r = A 



We may similarly find a number of steps that will guarantee that an approximate root has 
been reached and that the above algorithm will always produce subsequent Newton steps. 
We do this by observing that .5 A'||s(x)||Aq'Q^/^ < .5 — 6 implies that ,5Kh(x)aQ^ < 
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.5 — 8. And this implies that 8 < g{x, 1) < 1 — <5. Thus we shall count the steps for 
which ||•s(a:Jt)|| exceeds (1 — 28)/aKAQ^^^. As above we find that 



A{x) > 



8^{.5-8fA^ 

(l + 2hQ2 K^Qi 



Ai 



We now ’’count” the steps. Since A(xfc) is bounded away from zero, say by A, we have 
f{xo) - f{xk) > ^ Set 

^ ^ /(xo)-mm/ 

A 



Set 



^ f{xo)-minf 
min{A, Ai } 

If = N, and Xk is not an approximate root, we have a contradiction. 

REMARK 3.1 The results above are disappointing in that the cost of the algorithm is 
sensitive to the condition number Q, in spite of the fact that Newton directions are taken. 
(Recall that Smale’s global Newton method was not sensitive to the condition number). 
Indeed the gradient method is less sensitive to Q than this method. We may classify this 
algorithm as ’’greedy” because it is trying to decrease f at each iteration. On the plus side, 
the algorithm is easily implemented. 



The most robust result over the class of strongly convex functions is due to Nesterov. A 
simple, easily coded algorithm using combinations of gradient steps will drive f{xn)/f{xo) 
to less than e in less than 

. By the theory of Nemerovsky and Yuden, for some positive number c no algorithm 
can do better than the following number of steps for every strongly convex function of 
condition number Q: 

minin, y/Q) 1 

c ; In — . 

lnmin(n,y/Q) e 

For the case when n > y/Q we can assert that Nesterov’s method is to within a slowly 
changing multiplicative factor essentially optimal over the class of strongly convex func- 
tions. This method and its generalizations are being studied by Osman Guler at U. of 
Chicago, School of Business. 
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Consider the Smale global Newton method applied to minimizing strongly convex functions 
of condition number Q. This method which is sensitive to a Lipschitz constant for the 
Hessian would not be a candidate for optimality on the set of strongly convex functions. 
The reason is that there exist strongly convex functions with condition number Q that have 
Hessians with arbitrarily large Lipschitz constants. However for the multitude of natural 
problems with bounded Lipschitz 2nd derivatives, or bounded values of the constant K, 
we should expect better results for large condition numbers from the Smale global Newton 
method, than with an algorithm of the Nesterov type. 
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We now digress to the problem of linear inequalities. Let A be an m by n matrix of rank 
n, and b an m by 1 matrix. Denote the ith row of A by A*. Set 

R{x) = Ax — b. 

We seek a solution of the system of inequalities : 

Ri{x) <0 1 = l,2,...,m. 

Or, if this system has no solution, to establish its inconsistency. Let G(x) = max{Ri(x) : 
1 <i < m}. We assume that G is bounded below, and consequently, that G has bounded 
level sets. To ensure that this is the case, a phony half-space may be added to our system of 
inequalities. Let Aq = — Ro(x) = [Ao,a;] — b. Let = max{Ri(x) : 0 < 

i < m}]. Let z(b) = argmin G*. If the original system is consistent then for b sufficiently 
large G(z(b))< 0. 

We shall employ the penalty function: 

m 

^^[maa;(0, i?,(a;))]^ with p> 2 

j=i 

We note that for p > 2 , F is convex. If p=2, F' is continuous, and F” has jump 
discontinuities. If p > 3 then F'" is continuous. Our numerical experiments using F to 
solve inequalities gave most satisfactory results with p=2, even though more smoothness 
is obtained with larger values of p. Thus in what follows we shall take p=2. We assume 
that F has bounded level sets. 

Let /+( x) denote the set of all indices from {1,2,3, ...,m} for which Ri{x) > 0 when 
* G /+( x). If dim{Ai : i G < n, then F"(x) does not have an inverse. We seek the 

following construction. 

CONSTRUCTION B 4.0 

Assume that at Xk the dimension of the set of gradients of the actice residuals is less than 
n. That is, dim {A,- 1 n Take h 0 such that [A,, /i] = 0, 1 < * < 5 . 

Let €k = .5{F{xk-\ — F(xk). Find the smallest t such that ^{Rj(x -|- th) : q + 1 < i < 
m and > 0} = e|/2(m — q) Set xjtj = xjt -|- th. Repeat this process if necessary. After 
at most m-q steps we obtain xjt+i. 
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ALGORITHM 5.0 Take xq arbitrarily in En- If dim{Ai : i € /'^(a;o)} < n use (4.0) 
to find xi with dim{Ai G = n. At the kth iteration we are given Xk and a 

non-singular H(xk). Choose 7 *; such that .75 > g{xk,yk) > -25, taking 'yk = 1, if possible. 
Set = Xk — ')fkH~^(xk)^ f(x). If H has an inverse at then set a:*r+i = Sjt+i 

Otherwise update Xk=i with the construction (4.0) or (4.1) to obtain Xjt+i. 

CLAIM 5.1 Assume the hypotheses of (4.0). The algorithm is well defined and will 
terminate in a finite number of steps, with a positive value of F if the system is inconsistent 
and with F = 0 if the system is consistent. For the latter case the argument of F is a 
solution of the inequalities. 

PROOF Let X = {a: G : [A*, a:] < b{, 1 < i < m}. Assume that X is not empty. 
Let w denote the number of vertices of the solution set X, and let Vg , s=l, 2 ,...,w denote 
the vertices. Each Vg lies in the intersection of at least n hyperplanes supporting X. 
Any point in X is on the right side (Ri(x) < 0) of these hyperplanes. Stated otherwise: 
= bi, i G Ig, cardig > n. Let C(vg) = {x £ En : [A\x] > bi,i G Ig} = {x £ En : 
[A‘,a: - Vg] > 0,i £ 7s},and let C(vg) denote its closure. Let Bg be a ball of radius Vg 
centered at Vg. If some hyperplane [A*, a;] — 6 ,- such that i £ I Ig meets C{vg) then there 
exists a minimal value of Vg such that Bg H C{vg) meets the closest hyperplane [A*, x] — 6 , 
such that i £ I ^ Ig. Points in Bg D C(s) ~ {u^) are on the wrong side of the same 
hyperplanes, n or more in number. In all the above sets s is understood to range over 
1 , 2 ,... ,w. 

It follows from the proof of (3.0) that the above algorithm generates a sequence {xjt} G S 
(the level set of F at xq ) such that F(xk) converges downward to min F. We claim 
that the only cluster points of {x;t} the vertices Vg, s = The sets I~^(xk) 

are collections of n or more out of m indices that are repeated infinitely often. Thus at 
least one of the collections must be frequently repeated. Let {x^,. } denote a subsequence 
converging to a limit v. Take a thinner subsequence if necessary such that only one index 
set is represented. Since F(v) = 0, it folows that v £ X. Since {xjt,- } is on the wrong side 
of at least n hyperplanes at points arbitrarily close to X, the index set I must be one of the 
sets Ig, s=l,2,...,w. Let k denote the least value of k such that Xj^ G R 3 H C'(s) for some 
s =1,2, ...,w. At most one more step at xj. terminates the process, because a Newton step 
minimizes the restriction of F to H C(vg) -a quadratic function,- in one step. 
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Assume now that X is empty. A necessary condition that a system of inequalities be 
inconsistent is that 0 belongs to the convex hull of the rows of the matrix A. Let x* 
minimize F. We claim that x* is unique, card /"•‘(x*) > n + 1, and 0 belongs to the convex 
hull of the rows of A. If card /"’‘(x*) < n choose h so that [A*, h\ = — 1, i Then 

for some h sufficiently small F(x*) can be decreased. Since x* is a strict local minimum 
it is unique because of the convexity of F. 

Let B be a baill centered at x* with the property that = /"'‘(x*) for all x £ B. This 

existence of such a ball follows from the continuity of i?i(x) for each i, 1 < * < m, and 
because R{{x*) ^ 0 for all i E I'^(x*). Because x* is unique the sequence {xfc} converges 
to it. Thus for some least k = ^ xj. E B. Thus the solution occurs here or at the next 
step. 

Notice that the parameter K of (3.0) is actually finite. Consider the denominator of K 
for k=0,l,2,...,^. Since ||■s(x)|| > ||V/(x)||/Q£ the denominator is always positive. (If 
V/(x) = 0 we have a solution) One more step at Xj^ terminates the process. The Newton 
step minimizes the quadratic function (restriction of F to H C{vg) in one step. Note 
that the numerator of K for this last step is 0, while the denominator is positive. 

NUMERICAL COMPUTATIONS 6.0 

The Blair example may be found in Kortanek and Shi. The coefficients for the case n=8 
are written out (Example 2a, p55) and a program is given to generate the coefficients for 
any n. For n = 12, the case for which we offer extensive calculations, the Blair problem is 
a linear programming problem (LP) of the form: minimize L(x) subject to 

Ri{x) <0 1 < z < 24 

of which 12 inequalities constrain x to lie in the first orthant. The coefficients of the 
system are integers between 1 and 305,175,780. Thus the system is very poorly conditioned. 
The problem has a unique solution x,- = 0, i=l,2,...,ll, Xj 2 = 1, with L(x) = 305175780. 
The system we solve contains following inequalities : 

r,(x) < 0, i = l,2, ...,25 where 

ri4(x) = L(x)- 305175780, 
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ri{x) = i2,(x), i = 1, 13 and 
rj+i(x) = Ri{x) i = 14, ...24 



Superlindo required 2438 iterations to solve the above LP problem with full accuracy. Be- 
cause one cannot enter linear inequalities directly into Superlindo we entered the following 
problem; min L(x) subject to 



ri(x) <0 1 < i < 25. 

Thus when phase I was finished the problem would terminate. This took 2236 iterations. 
Let 

r(x) = Ax — b. 

The components of A and b are listed at the end of this section. 

A simple scheme of row scaling was used to convert to a better scaled problem. Let bm 
be the maximal component of the right hand side. bm=305175780. The scaled system is: 

(6m/|6,|) r;(x) < 0 i = 1, 2, 3, ..., 14 

bm r,(x) <0 i = 15, ..., 25 

Thus 4.0 should be used. Some computations were made before 4.0 was coded using an 
ad hoc procedure of an overrelaxed gradient step to move off a degenerate point x. The 
step-length chosen was 100/||/7(x)||. This has worked for this problem in at most 3 steps 
for every example (over 100) tried. However the penalty function usually increases during 
these perturbations. Subsequently, the method of 4.0 was coded. The results were similar- 
to those given below, with the moves in the null spaces (which we think of as gear shifts) 
replacing the gradient steps. 

There follows 15 successive runs with random initial vectors with components lying between 
-1000 and 1000. The 12 numbers after xo are the components of the starting vector. 

The last column contains a list of integers in order 1,2,3 interspersed with the symbol *. 
Each integer numbers a normal step of the algorithm while * indicates a gradient step. 
In what follows disregard this last column. We explain first the lines containing tt, info. 
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rmax, number, number, number. Here tt means the number of active residuals; its value 
is the first number. Info is a test for degeneracy; its value is the second number. If info = 
0 the point is nondegenerate, info = 1 , degenerate. Rmax is the value of the maximum 
residual. It is the 3rd number. 

Now, the line that contains step-length, gamma, number, number. Here step-length = 
\\H-^ (a;)V/(a:)||. Its value is the first number. The meaning of gamma is that of algorithm 
3.0, the fraction of a Newton step. The value of gamma is the second number. At the 
bottom of the page we have x,r number ,number. This the row- wise print out of the solution 
vector followed by the residuals at solution. 
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APPENDIX THE COMPONENTS 


uF THh BLnlR flA TRICES A 


■1-^ D 


FIFTEEN RUNS WITH RANDOM 


STARTING POINTS 




a(i, 1)=“5. 


a < /■■ , S ; = 0 . 


a V 13, 5 > = 0 . 


a ( 2. 1 ; =- 1 . 


a 8 . 3 ; =0 . 


a u 1 ^ . 5 ) = 3 3 G 5 . 


a ( s , 1 ) = 0 . 


a(3« 3) — 0* 


a < 1 5 . 5 ) = 0 , 


a ( -+ , 1 ) =0. 


a C 10, 3) =0. 


a V 1 6 , 5 ) = 0 . 


a ( 5 . 1 ) =0. 


a(ii,3>=G. 


a ( 17 , 5 J =0 . 


a(6. 1) =0. 


a ( 12, 3) =0. 


a ( 1 3 , 5 ) = 0 . 


a ( T , 1 ; = 0 . 


ac 13, 3; =0. 


a(lS,5>=— 1. 


a(8, 1) =0. 


a(14,3)=lS5. 


a(.i,0, S J = o. 


a ( 9 . 1) = 0 . 


a'.15,3;= 0. 


a i 2 1 , 5 ) = 0 . 


a ( 1 0 . 1 .J = 0 . 


a ( lb, 3 ) = 0 . 


a ( 2 2 , 5 ) = 0 . 


a (1 1 . 1 ) =0. 0 


a(17,3)=“'l. 


a 23, 5 ) =0 . 


a( 12, 1 ) =0. 


a( 13, 3) =0. 


a(24, 5‘)=0. 


a ( U , 1 j = 0 . 0 


a C 1 9 , 3 ) = 0 . 


a ( 2 5 , 5 > = 0 . 


a(l4, 1;=5. 


a ( 20 , 3 ; =0 . 


a( l,b> = -180. 


a(l5,l) =“1. 


aC21 , 3) =0. 


at.2, 6) = -30. 


a t 1 6 , 1 .1 = 0 . 


a (22, 3) =0, 


a ( 3 , 8 ) = - 4 0 . 


ailT, 1^—0, 


3 j = 0. 


a ( a, 6 ) = -20. 


a C 1 S . 1 .) = 0 . 


a C 24 , 3 } = 0 . 


a ( 5 , S = - 1 0 . 


a t 1 9 . 1 ) = 0 . 


a C 4 5 ♦ 3 ) = 0 . 


ai.6, b; = -5. 


a ( 20 , 1 ) = 0 . 


a( 1 , 4 ) =-40, 


a(7,6)=-l. 


a ( 21 , 1 ) =0, 


a ( 2 , a ) = “20. 


a ( 3 , 6 ) = 0 . 


a(22, 1 ) =0. 


a ( 3 , 4 ; = “ 1 0 . 


a ( 9 , 6 j = 0 . 


a C 23, 1 ) =0 , 


av.4,4)=-5. 


a ( 1 0 , 6 ) = 0 . 


a ( 24 , 1 J = 0 , 


a ( 5 , 4 ) = - 1 . 


a ( 1 1 , 6 ; =0 . 


aC25, 1>=0, 


a ^ 6 , 4 ) = C . 


a ( 1 2 , 6 ; =0 . 


a( 1, 2) =-10. 


a ( 7 , 4 ) = 0 . 


a(13,6)=0. 


a(2,2)=“5. 


a(3, 4) =0. 


a ( 1 4 . 6 ) = 1 9530 


a ( 3. 2) =- 1 . 


a(9, 4; =0, 


a ( 1 5 , 6 ; =0 . 


a c 4 , 2 ) =0 . 


a ( 10, a ) =0. 


a(16,6)=0. 


a (5. 2) =0 


a ( 1 1 , 4 ) =0. 


a(l7,6)=0. 


a (. 6 , 2 ) =0 . 


a( 12, 4}=0. 


a( 18, 6 ) =0. 


a<7, 2) =0. 


a ( 13, 4 ) =0. 


a(19,6)=0. 


aca, 2) =0. 


a C 14, 4) =730. 


a (20, 6 J =-i . 


a ( 9 . 2 ) =0 . 


a( 15, 4) =0. 


a(2l,S)=0. 


a ( 1 0 , 2 .) = 0 . 


aC16,4)=0. 


a(22, S> = 0. 


a ( 1 1 . 2) =0. 


a( 17, 4) =0. 


a ( 23 , 6 ; = 0 . 


a ( 12, 2 ) =0. 


a(13,4)=”l. 


a C 24 , 6 J =0 . 


a ( 13. 2 ) =0, 


a( 19, 4) =0. 


a ( 25 , S ) = 0 . 


aCl4,2)=30. 


a 20 , 4 ) =0 . 


aCl, t J — ~ 3 . 


a ( 15. 2) =0, 


a(21 , 4) =0. 


a(2,7;=-lb0. 


a t IS . 2) =- 1 . 


a ( 22 , 4 ) = 0 . 


a(3, 7) =-80. 


a( IT, 2)=0, 


a ( 2o , 4 ) = 0 . 


a(^, < )=-»4w. 


atl3,2j=0. 


a(24, 4) =0. 


a(5, 1 ) = — ..i-G. 


a(19,2;=0. 


a(25,4)=0. 


a(6,7)=-i0. 


a ( 20 . 2 ) =0 . 


a 1 , 5 ) = - 60 . 


a(7,7)=-5. 


a t 21 , 2 ) =0 . 


a ( 2, 5 ) = -40. 


a(3, 7) =“ 1 . 


a(22, = 0. 


a(3, 5; =-20. 


a ( 9 , 7 ) = 0 . 


a(23, 2) =0. 


a(4, 5) =-10. 


a ( 1 0 , 7 ; =0 . 


a C 24. 2 ) =0 . 


a(5, 5) =-5. 


a<.il,7)=w. 


a ( 25, 2; =0, 


a ( 6 , 5 ) = - 1 . 


a( 12, 7 ) =0. 


a( 1 . 3) =-20. 


a 7 , 5 ; = 0 . 


a ( 1 3 . 7 ) =0 . 


a(2,3j=-l0. 


a ( 3 , 5 ) = 0 . 


a C 14, 7 / = 9 785b 


a C 3 , 3 ) = -5 . 


a ( 9 , 5 ) =0 . 


a ( 1 5 , 7 ) = 0 . 


a(4, 3) =-l . 


aC10,5)=0. 


a ( 16 , 7 ) =C . 


a ( 5 , 3 ; = 0 . 


a ( 1 1 , 5 ) =0. 


a( 17, 7 >=0. 


a(6, 3; =0. 


a( 12, 5) =0. 


a(lo,7)=0. 
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A i y 4 


a 2 5 . 9 ) = u 


il\. — <J ^ W. 


a >w 1 , 1 0 J = 


=t(^i,7^=“l. 


ac2, 10v'=- 


a(22, 7) =0. 


a C 3 , 10)=“ 


a c 23 , 7 ) =0 . 


av.4, 10/ = — 


a(.i-4,7)=0. 


a ( 5 . 10)=- 


a C 2 b , / > = 0 . 


a^6, 10;=- 


aC 1,6) =“640. 


a ( 7 , 10;=- 


a'v2,d;=“320. 


ac3, 10;=- 


a 3 , S ; = - ISO . 


a(9, 10)=- 


a 3) =-30. 


aClO, 10)= 


a(5, 8-> = -aO. 


a ( 1 1 . 10) = 


av6, 3) — “!l_0. 


a 1 2 , 1 0 ) = 


a 7 . 8 ; = - 1 0 . 


aC 13, id) = 


a 3 , =3 ) = - 5 . 


aCl4, 10)= 


aC9, h*)=“l. 


a ( 1 5 , 10)= 


a( 10, d) “O* 


aCl6, 10)= 


a c 1 1 , a ) =0. 


aCl7, lO)= 


a t, 1 2 , 3 ) = 0 . 


aCl3, 10>= 


a(13.&)=0. 


a c 19 , 10 ) = 


aC 14, 3>=4.38230. 


aC20. 10) = 


a C 15 , 8 ) =0. 


a C21 , 10) = 


a 1 lb, 3 ) = 0 , 


a ‘ 22 , 10)= 


a(17,S)=0. 


a c23, 10 ) = 


aC la, 3) =0. 


a C 24 , 10) = 


aC 19, 3) =0. 


a c 25 ,10) = 


a C 20 , 8 ) =0 . 


a C 1 , 11)=- 


a ( 21 , S ) =0. 


a(2, 1 1 ) =- 


a c 22 , 3 ) = - 1 . 


aC3, 11)=- 


a(23, tj ) = o. 


a ( 4 , 11)=- 


a(.il4, 8) = 0. 


ac5, 11)=- 


a(.:l5, 3) = 0. 


a^6, 11)=- 


a C 1 , 9 ) = - 1230. 


a(7, 11 ) =- 


aC2, 9) = -6 m-0. 


ac8, 11)=- 


a i. 3 , 9 ) = - 320 . 


a^.O, 11;=- 


a 4 , 9 > = - 160 . 


a ( 10, 11) = 


a t 5 , 9 ) = -80 . 


a i 1 1 , 11) = 


a C 6 , 9 ) = - -40 . 


a ^ 12, 1 1 ; = 


a (l 7 , 9 ; = -20 . 


a C13, 11; = 


a c S , 9 ; = - 1 0 . 


a ( 14 , 11) = 


a 3 , 9 ; = - 5 


a 15 , 11) = 


a( 10. 9) =-l . 


a ( 16, 11) = 


a^ll,9)=0. 


a L 1 7, 11) = 


av. I.L., 9) “~1. 


a C13, 11) = 


ac la, a) =0. 


a ^ 1 9 . 11) = 


a 1m-. 9 ) = 2441405. 


a (20, 11) = 


a c 1 5 , 9 ) = 0 . 


a ( 21 , 11)= 


a c 1 6 , 9 ) = 0 . 


a ( 22 ,11)= 


acl7.9)=0. 


a(23, 11;= 


a ( 1 6 , 9 >> = 0 . 


a(^4, 11)= 


a V. 1 9 , 9 ) = 0 . 


a <4 25 , 1 1 ) = 


a ( 20 . 9 ; =0 . 


a( 1 , 12; =- 


a(^l, 9) =0. 


a<.2, 12;=- 


a ( 22, 9 > = 0. 


a(3, 12) =- 


a 2 a , 9 j = - 1 . 0 


a ( 4 , 12;=- 


a 9-) = 0. 


a(5, 12)=- 



I 





av.6. 12; = -3 20. 


- 2560 . 


a*.T, i^j=-loO. 


1 230 . 


av.8, 12;=“3U. 


640 . 


a(9, 12)= — 4o. 


320. 


aClO, 12) =-20. 


1 So . 


acll, 12;=-10. 


30 . 


a( 12, 12) =-5. 


40 . 


a ( 13, 12 ) =- 1 . 


20. 


a C 14, 12) =305175 


10. 


a(15, 12)=0. 


-5. 


aCl6, 12; =0. 


- 1 . 


aCl7, 12)=U. 


0 . 


ac 18. 12; =0. 


0. 


a(l9, 12;=0. 


Ij^-x-O f O30. 


a 20 , 1 2 ; =0 . 


0. 


a ( 2 1 , 1 2 ) =0 . 


0 . 


a(22, 1^)=0. 


0. 


aC23, 12;=0. 


0. 


a^.24, 12;=0. 


0 . 


a(25, 12) =0. 


0 . 


b ( 1 ) = -4096 . 


0. 


b ( 2 ) = -2043 . 


0. 


bC3) =—1024. 


0. 


bC4) =-£12. 


-1 . 


b(5)= -256. 


0 . 


b(6) =-128. 


5120. 


b ( 7) =-64. 


2560. 


b (3) = — o2. 


1230. 


b ( 9 ) = - 16 . 


640 . 


b C 10; =-a. 


320. 


b ( 1 1 ) = - 4 . 


160. 


b ( 12; =-2. 


80. 


b 13 ) = - 1 . 


40 • 


b( 14) =305175730 


20. 


b ( 1 5 ; =0 . 


-10. 


b ( 1 6 ) = 0 . 


-5. 


bt 17) =0. 


- 1 . 


b ( 13) =0. 


0 . 


be 19) =0. 


61035155. 


b (20) =0. 


0 . 


b(21) =0. 


0. 


b (22) =0. 


0 . 


b(23) =0. 


0. 


b (24) =0. 


O • 


b ^ 25 ) = 0 . 



0 . 

0 . 

0 . 

0 . 

0 . 

-i . 

10^40 . 
5120 . 
2560 . 
12 - 30 . 
640 . 
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RUfj 

kO 



tt , 

1 1 , 

5 t e 
1 1 , 

s t e 
1 1 , 

3 t. 0 

t t , 

3 t 0 

t t, 
St0 

t t , 

tt , 
tt , 

s t e 
1 1 , 

s t e 
1 1 , 

S t 0 : 

t t, 

3 t e : 

t t , 

s t e; 
1 1 , 

s t 0 1 
tt , 

s tei 
1 1 , 

X 



4L li [Niewton steps and a t^radierit steps, 

- 9 7 i . 9 7 9 Z 6 6 3 9 1 7 d £ 
593. 593-439850*4.01 
a25. 520516159178 
268. 5973^8237653 



“157. 4-996.-6709990 
£• 07. 770637071353 
•877. 746807236177 
394. 635221023124 



“743. 32-^ 1 03d3853S 
5dl . 2 01492434374 
“361. 133546071757 
563. 4129^9712823 



info, rmax , 
into, r max , 


7. 00000000000000 

14. 0000000000000 


1 296624527029.611 

1 9653257644971.96 




p 1 en g t h , gamma 
i i'l to. 1 ma X , 


47765. 5239230591 
l4. 0000000000000 


1 . 00000000000000 

0 2357700197.06739 


1 


p 1 0 ri g t ti , gamma 
1 1*1 r o , i nia X , 


19. 2575641 130305 
12. 0000000000000 


1 . 00000000000000 

0 2428295204. 71995 




p 1 eng t h , gamma 
i ri r o , X' m a X , 


5454. 13223736253 
l4. 0000000000000 


0. 305175781250000D“00 

0 2428421099.03133 


3 


p 1 eng t h , gamma 
i 1*1 f o , 1 m a X , 


17. 3705077615239 
12. 0000000000000 


1 . 00000000000000 

0 1 185139263. 73383 


4 


p 1 eng th , gamma 
into, r ma x , 


5472. 94705380250 
13. 0000000000000 


0. 762939453125000D-00 

0 1185130226. 38878 


5 


info, rmax , 
i ti f o , rmax. 


1 . 00000000000000 
19. 0000000000000 


1 620675091,14546 

1 163G622909S9. 794 


* 


p I eng th, gamma 
info, rmax. 


198. 732501363480 
13. 0000000000000 


1 . 00000000000000 

0 153611593. 072991 


6 


piength, gamma 
info, rmax. 


196. 752514409908 
14. 0000000000000 


0. 152587890625000D“00 

0 153610594. 293101 


7 


piength, gamma 
info, rmax. 


66. 3907250047904 
13. 0000000000000 


0. 107421675000000D“001 

0 1 5 29 48384 . 53 7403 


a 


P I eng t h , gamma 
i ri f o , rmax. 


S3. 3248672145210 
16. 0000000000000 


0. 131835937500000D“001 

0 152548504. 421150 


9 


p I eng th, gamma 
info, rmax , 


0. 584973381718277 
12. 0000000000000 


1 . 00000000000000 

0 133329412. 581193 


10 


p I eng t h , gamma 
i nf o , rmax , 


0. 504481166507457 
12. 0000000000000 


1 . 00000000000000 

0 0. 250339508056641D 


1 1 

“005 



P I eng t h , gamma 0 . 9365949088823370“ 0 1 4 1 . 00000000000000 

info, r max, 12.0000000000000 0 0.000000000000000 



, r 0. 1656299752173 
Q. 1756448109281 16D“030 
9. 173795918181504D“029 
0. 1 360 78506 1 50625D “ 028 
1 . 00000000000000 
“4577 6 3664. 000000 
“457763664. 000000 
“457763664. 000000 
“457763664. 000000 
0. 505a62562l5S010D“G23 
0. 126012992240322D“021 
0. 105324292021762D-020 
0. 818520212283408D-020 



D-031 0. 60859386242636 

0. 412919380076623D“030 
0. 345 1266460341930“ 029 
0. 26S212707775144D“028 
“457763664. 000000 
“457763664. 000000 
“457763664. 000000 
“457763664. 000000 
0. 000000000000000 
“0. 185723104234804D“022 
“0. 259549148196485D“021 
“0. 2 1 C6485o4043524D“020 
“0. 1661 1 1454645750D-019 



6D-031 

0. 85049066344 1 403D “ 030 
0. 690253292068385D“029 
0. 5443 1 402 4602 4980 “028 
“45776o664. 000000 
“457763664. 000000 
“457763664. 000000 
“457763664. 000000 
0. 000000000000000 
“0. 536025414753610D“022 
“0. 53038 30 4 1 966 7 30D “ 02 1 
“0. 415273637 1 l4376D“020 
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RUN 



N e w t o fi 5 h e p 3 f* n H 



d i' U 1 n t 5 t 8 p 5 



xO 



-93G. 292890085242 
2aa. 623299272343 
-508. 577056980336 
416. 323740033475 



•363. 844227490599 
719. 519989562319 
-162. 2^6891100147 
-703. 495647012945 



1 1 , info, r m a X . 
1 1 , into, f ma X , 
1 1 . i I'l t o . f m a X , 



5 t e p 1 e n g t h , g a m m i 
1 1 , Into, r m a X , 



5. 00000000000000 

1 . 00000000000000 
16. 0000000000000 



s t e p i e ng t h , gamma 
1 1 , info, f iTi a X , 



435723. 172539526 
0000000000000 



10514. 7126566537 
1 4 . 0000000000000 



a t a p i tf n g t h , gamma 
1 1 , i t o , r ma X , 



60. 4 5 1 0 9 4 4 9 8 29 4 7 

14. 0000000000000 



t a p 1 a n g t h , gamma 
t t , info, rmax , 



52. 5504577006773 
15.0000000000000 



s tap 1 angth, gamma 
tt, into, riTiax, 



126. 039756342685 
14. 0000000000000 



a tap i ang t h, gamma 
1 1 , info, f ma x , 



0.649075995643773 

12. 0000000000000 



a tap I angth, gamma 
1 1 , into, rmax. 



0. 490203091522860 
12. 0000000000000 



-322. 317913607396 
543 . 25499804348 1 
618. 731966442102 
253. 0832207 13413 
283902654639. 046 
137 1 1 16280676. 87 
359774939314244. 



1 . 00000000000000 

0 161170658. 157925 



^46582031250000-003 
0 161126391.314559 



0. 3125000000000000-001 

0 156091441 . 193965 



0. 125000000000000 

0 148358483. 404517 



0. 6445312500000000-001 

0 147307230.642514 



1 . 00000000000000 
0 12955577^ 



393379 



1 . 00000000000000 

0 0. 244379043579102D-0( 



s tap i angth, gamma 
1 1 , into, rmax. 



0. 9131 138868539100-014 

12. 0000000000000 



1 . 00000000000000 
0 0. 000000000000000 



X , r O, 331259950434605D-031 0 . 1 00 1 48357 108 1 36D -030 



0. 200296714216273D-030 
0. 1676329423594650-029 
0. 1341063538875720-028 

1 . 00000000000000 
-457763664. 000000 
-45 / 763664. 000000 
-^57763664. 000000 
-^57763664. 000000 
-0. 101092512431602D-022 
-U. 145 U 7^5967758420-021 
-0. 1030673429070100-020 
“O. o 06 -483 1503330640-020 



0.4098378921656040-030 
0. 3377310750477460-029 
0. 2642684032490390-028 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
0. 000000000000000 
-0. 305628525956006D-022 
-0. 2501451935516350-021 
-0. 2076393165571880-020 
“0. 1 6 1 296 6 3006 76 1 30 - 0 1 9 



0. 8196757843312070-0:: 
0. 6803925307531230-0: 
0. 5285368064980780-0: 
-457763664. 000000 
“457763664. 000000 
-457763664. 000000 
-457763664.000000 
0.000000000000000 
-0. 61 125705 19 1201 2D -O: 
-0. 51 1575 132677 130D-0: 
-0. 409260106141704D-0: 



22 



RUN 



[^1 e w t. o n steps end i gradient step 



kO 38a- . 636 1 360a03^ 

890. 55686a074000 
843. o9i)3 045G.:I4a8 
-9a. 61259al835709 



521. 53299147395' 
22. 8991 136970037 
823. 56359299-3691 
-466. 4&6a48766907 



-700. 058083640171 
-639. 569422^-39330 
996. 34 OI 1 329 757 4 
-450. 87S85 423801m- 



1 1 , i ri r o , r m a X , 



17. 0000000000000 



529467259301 , O 64 



stap'iengtl’i. garnrna 2212.2/366031^13 

tt, into, rmax, 14.0000000000000 



1 . 00000000000000 1 
0 194786857. 815711 



staple ngth, gamma 11. 5597435264857 

tt, into, rmax, 15.0000000000000 



0. 937500000000000D-001 

0 179212381 . 151034 



step length, gamma 17. 4100728479677 

tt, into, rmax, 15.0000000000000 



0.31250000 00 00000 D - 001 -3 

0 173957713. 602083 



steplength, gamma 
1 1 , into, rmax , 



13. 19102590098: 

16. 0000000000000 



0. 109375000000000 

0 156136399.016667 



steplength, gamma 
1 1 , info, rmax. 



4. 33788792413538 
0000000000000 



0. 375000000000000 

0 1 26699 425 . 005i 



steplength, gamma 0. 654804890385739 
t t , i nf o , r max , 12. 0000000000000 



1 . 00000000000000 

0 134111089.546327 



steplength, gamma 0. 5074379436994/4 
tt, inf o, rmax, 12. 0000000000000 



1 . 00000000000000 7 

0 0 . 2503395080566 4 1 D - 005 



steplength, gamma 
1 1 , i n f o , rmax , 



0. 93679698241 80 18D-0 14 

12. 0000000000000 



1 . 00000000000000 
0 0. 000000000000000 



: , r 0.13096323621833: 
0. 140207699951391D-030 
0. 173795918181504D-029 
0. 138050658413677D-028 

1 . 00000000000000 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
-0. 39966S072404008D-023 
-0. 1 13787851202082D-021 
-0. 1053242920217620-020 
-0. S1852021228340SD-020 



:0-031 0. 4853343459855830 

0. 3728600372333690-030 
0. 3451266460341930-029 
O. 2682127077751440-028 
-457763664. 000000 
-457763664 . 000000 
-457763664. 000000 
-457763664. 000000 
0. 000000000000000 
-0. 1 48 1 1 228565 5603D -022 
-0. 259549148196485D-021 
-0. 21 064S584043524D-020 
-0. 1661114546457500-019 



-031 

0. 8504906634414030-030 
0. 6902532920683850-029 
0. 5443140246024930-028 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
0. 000000000000000 
0. 427S7993633S40SD-022 
0. 530383041966730D-021 
0. 421297168037048D-020 
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FvUli 



/ 



f 'i e w t o II 



s t 0 P 3 



xO 



- d4^^3xi7477Z£ 



741 . 53360^-52957 1 



244. 333634286300 



341 . 271154817054 
4 6 . 8 1 3 / U 9 4 50808 
-966. 621403447874 


324. 279857033038 
60a. 307441254551 
-693. 073136470057 




112. 113379925855 
-321 . 373886549000 
-623. 141231756370 


i n r Q , r max, i 7 . 


0000000000000 


0 


1 096 22 73986 23 . 35 



5 t e p I e n g t h , gamma 
1 1 , 1 nr u , rmax » 

step length, gamma 
1 1 , t ni o , 1 * max , 

step I eng th, gamma 
1 1 , info, f m a X , 

steplength, gamma 
1 1 , i i"i t o , rm a X , 

steplength, gamma 
1 1 , i n r o , rmax , 

steplength, gamma 
1 1 , i n 1 o , rmax, 

s t e p I en g t h , gamma 
1 1 , into, rmax, 



2002. 309682 /693S 

13. 0000000000000 

130. 062162441505 

14. 0000000000000 

29. 8103139169651 

15. 0000000000000 

7. 61286793105905 

16. 0000000000000 

0. 480797575295140 
12. 0000000000000 

0. 504481166507457 
12. 0000000000000 

0. 918104743739594D-014 
1 . 00000000000000 



1 . 00000000000000 

0 153540383. 712848 

0. 634765625000000D-002 

0 153117350. 272783 

0. 253906250000000D-001 

0 151769052.907618 

0. 937500000000000D-001 

0 149354307. 884913 

1 . 00000000000000 

0 133329412. 531193 

1 . 00000000000000 

0 0. 244379043579102D-0 

1 . 00000000000000 

1 0 . 596046447753906D- 0 



0. 770371977754894D-033 0. 433334237487 1 28D-032 

0. 9629649721936180-031 
0. 9170508023194260-029 



X , r 

0. 2060004339938210-031 
0. 2021456069628840-029 
0. 1830157300112750-027 
1 . 00000000000002 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
-0. 235093366 120005D-024 
-0. 2338735o2650006D- 022 
-0. 2793616902292530-020 
-C. 2426671683181400-018 



0. 7951717924627800-027 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
0. 5960464477539060-007 
-0. 1322431 12192503D-023 
-0. 1344765514206430-021 
-0. 1251354442315800-019 
-0. 1032298452432720-017 



0. 4406527712753000-0 
0. 4102076707149260-0' 
0. 3382635561587700-0 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
-0. 5960464477539060-0' 
-0. 6347669385240120-0 
-0. 6168994246988920-0 
-0. 5585196742689730-0 
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RUN 



liewion steps 



kO - 1 i 4 . 1 477 1 76582 i 6 
624. m-6 1 6624 1 9657 
l=.*3. 737734.1:0 9 1593 
-427. 42064545a674 



553. 470022662264 
'654. 960097720481 
675. 86088001 1940 
*607. 602961981098 



-490. 039243319173 
-156.588812028271 
297. 942673453505 
-603. 144542822476 



1 1 , info, r 111 a X , 



19. 0000000000000 



762403473253. 473 



s t e p I e n g t ti , gamma 
1 1 , i n f o , r ni a X , 



1696. 80530559346 

13. 0000000000000 



1 . 00000000000000 

0 1 9 466448;! 



.932851 



step length, gamma 78. 7277480487342 

1 1 , into, r ma x , 15. 0000000000000 



0. 122070312500000D-003 

0 194642039.668371 



staple ngth, gamma 10.5301245257422 
tt, info , 1 max, 16.0000000000000 



0. 125000000000000 3 

0 171746426. 699385 



staple ngth, gamma 
1 1 , info, r m a X , 



8. 22887713006761 

16. 0000000000000 



0. 156250000000000D-001 

0 169258335. 804595 



steplength, gamma 
1 1 , info, r m a X , 



4 . 4053446 1 1 34:; 

IS. 0000000000000 



1 1 



0. 312500000000000 5 

0 125867764.378846 



steplength, gamm 2 
1 1 , info, rmax , 



0. 6 370372308565 29 
12. 0000000000000 



1 . 00000000000000 6 
0 134638584.993735 



steplength, gamm^ 
1 1 , info, rmax. 



0. 509430809427019 
12. 0000000000000 



1 . 00000000000000 7 

0 0. 250339508056641D-005 



stepl ength, gamma 
1 1 , info, rmax. 



0. 93674580055 1893D-01 4 

12. 0000000000000 



1 . 00000000000000 
0 0.000000000000000 



: , r 0. 108815041857879D 
0. 1 17866912596499D-030 
0. 173795918181504D-029 
0. 138050653413677D-028 

1 . oooooooooodooo 

-457763664. 000000 
“457763664. 000000 
-457763664.000000 
-457763664. 000000 
-0. 332077148394506D-023 
-0. 9497994 19 1248 19 D- 022 
-0. 106828924764930D-020 
-0. 81 85202 122S3408D-020 



031 0. 3967415685437- 

0. 31 123027901 29 77D- 030 
0. 350057026691824D-029 
0. 2682 1 2707775 1 44D - 028 
-457763664.000000 
-457763664. 000000 
-457763664.000000 
-457763664. 000000 
0. 000000000000000 
0. 121075916051802D-022 
0. 2332 18075 19 1045D -021 
0. 213657S49529860D-020 
0. 1661114548457500-019 



lD-031 

0. 764209001932355D-0 
0. 700114053383648D-0 
0. 5 443 1 402460249SD - 0 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
0. 000000000000000 
-0. 359701265163607D-022 
-0. 5303830419667300-021 
“0. 42 1 29 7 1 63037046D - 020 
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CO C4 C sl 



RUN #6 



f sl e w t o n steps and 2 gradient steps 



xO 



472. 36107968i0a8 
399. 660706261435 
64. 0326036041202 
955. 477280014122 



-831 . 003678005437 
814. 7354102385^8 
-59. 80632686800 1 8 
640. 367128180124 



-740. 935969423871 
- 6 5 2 . ^ 8 8 6 9 2 0 5 1 4 2 5 
72. 9986403563025 I 
712. 58522996 i 250 



1 1 ♦ info, r max , 5 . 00000000000000 

1 1 , into, 1 ma x , 16. 0000000000000 



26808381 1943.665 
23715049130938. 6 



step length, gamma 50637 . 9972940535 

1 1 , 1 n f u , i max , 12. 0000000000000 



1 . 00000000000000 

0 161169141, 153651 



staple ngth, gamma 10514. 7126636119 

tt, into, rmax, 14.0000000000000 



0. 274658203125000D-003 

0 161124874. 726943 



steplength, gamma 60.4510874755542 
tt, info, rmax, 14. 0000000000000 



0. 312500000000000D-001 

0 156039971 . 999712 



steplength, gamma 52. 5504645937000 
tt, info, rmax, 15. 0000000000000 



0. 125000000000000 

0 148353479.963610 



steplength, gamma 126.089752134237 

tt, into, rmax, l4. 0000000000000 



0. 644531250000000D-001 

0 147307227.428061 



steplength, gamma 0. 649075033909 147 
tt , 1 nf o, rmax , 12. 0000000000000 



1 . 00000000000000 

0 129555772. 393379 



steplength, gamma 0. 490203091522860 
tt, info, rmax, 12. 0000000000000 



1 . 00000000000000 

0 0. 244379043579 102D- 



steplength, gamma 0.9119245 46 507179D-014 1. 00000000000000 

tt, info, rmax, 12. 0000000000000 0 0. 000000000000000 



X , r 0. 331259950434605D-031 0 . 986076 1 3 1 526265D - 03 1 



0. 200296714216273D-030 
0. 167632942359465D-029 
0. 134106353887572D-02S 
1 . 00000000000000 
-457763664. 000000 
-45776366U. 000000 
-0-5 / 76366a. U 00000 
-a5776366a. 000000 
-0. 1010Q2512a31602D-022 
-0. 12al3220131 1362D-021 
-0. 101 5 647 10163642 Ei - 020 
-0. 806483 1 5033606aD- 020 



0. 406756404254534D-030 
0. 332600694390114D-029 
0. 26426S403249039D-028 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
0. 000000000000000 
-0. 3009265486336060-022 
-0. 250 145 193551 6850 -021 
-0. 2046300530708520-020 
-0. 1583692176735440-019 



0. S19675734331207D- 
0. 6705317694378600- 
0. 5206481974453680“ 
-457763664. 000000 
-457763664. OOOCOO 
-457763664. 000000 
-457763664. 000000 
0. 000000000000000 
-0. 61125705191.^01.-Lj" 
-0. 51 15751326771 30 E»- 
-0. 409260106141 / GaL)- 
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RUN #7 



8 Newton steps 



xO -980.614385562064 
872.092335185815 
-787. 423383854608 
-801 .412660064975 



754.663860379469 
903.372305432807 
-889. 882406466205 
-257.581608353113 



-849.966118423708 
903.822716644623 
-656. 189370753424 
-733.888854017488 



1 1 , info, r max , 



20. 0000000000000 



930810709180. 125 



step length, gamma 
1 1 , Info, rmax , 



2781 . 62252337554 
1 2 . 0000000000000 



1 . 00000000000000 

O 153611829.496706 



steplength, gamma 
1 1 , Info, rmax , 



3648. 13836926195 
15.0000000000000 



O. 391006469726563D-004 

0 153605823.174790 



steplength, gamma 
tt, info, rmax. 



64. 0782140937533 
16. 0000000000000 



0. 830078125000000D-002 

0 153096180.228943 



steplength, gamma 
1 1 , info, rmax , 



27.6115425905722 
1 5 . 0000000000000 



0. 195312500000000D-001 

0 152568067.708317 



steplength, gamma 
1 1 , info, rmax , 



73.2258353230729 
15 . 0000000000000 



O. 305175781250000D-004 

0 152567395.728954 



steplength, gamma 
1 1 , info, rmax , 



0.340599172069112 
12. 0000000000000 



1 . 00000000000000 

0 132047268.790020 



steplength, gamma 
1 1 , info, rmax , 



O. 499630146021219 
12.0000000000000 



1 . 00000000000000 



O. 250339508056641D-005 



steplength, gamma 0.934835255493608D-014 
tt. inf o, rmax, 12.0000000000000 



1 . 00000000000000 8 
0 0.000000000000000 



X . r 0. 215704153771370D-031 O. 801 186856865090D-031 



O. 206459690038312D-030 
0. 168865537523873D-029 
0. 136078506150625D-028 
1 . 00000000000000 
-457763664. 000000 
-457763664.000000 
-457763664. 000000 
-457763664.000000 
-0. 658276825 13601 3D-023 
-O. 126012992240322D-021 
-0. 1045719756501780-020 
-0. 8 185202 122834080-020 



O. 4129193800766230-030 
0. 3426614557053770-029 
0. 2682127077751440-028 
-457763664. 000000 
-457763664.000000 
-457763664.000000 
-457763664.000000 
0.000000000000000 
-O. 244502820764605D-022 
-0. 257668357267525D-021 
-O. 207639318557188D-020 
-O. 163704042456682D-019 



O. 844327687619364D-030 
O. 680392530753123D-029 
O. 536425415550288D-028 
-457763664.000000 
-457763664.000000 
-457763664. 000000 
-457763664.000000 
0 . 000000000000000 
-O. 63006496 120 161 2D-022 
-0. 515336714535050D-021 
-O. 4152786371 14376D-020 
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RUN #8 



9 Newton steps 



xO 



489. 859138456204 



20.9052331088909 



174. 179628824158 -653. 211700218227 
425.973625697280 -171. 141846780673 
955. 040074874622 -896. 536088135900 


1 1 , info, rmax , 


18. 0000000000000 


step length, gamma 
tt, info, rmax, 


2023.42010222668 
12. 0000000000000 


steplength, gamma 
tt, info, rmax. 


15612. 1105994219 0 

14. 0000000000000 


steplength, gamma 
tt, info, rmax, 


23.5758796432499 0 

16.0000000000000 


steplength, gamma 
t t , info, rmax , 


18.9507539629753 0 

15.0000000000000 


steplength, gamma 
1 1 , info, r ma x , 


34.1524046774300 0 

16.0000000000000 


steplength, gamma 
1 1 , info, rmax , 


92.0011728271753 0 

14.0000000000000 


steplength, gamma 
t t , info, rmax , 


0. 213780894549538 
12. 0000000000000 


steplength, gamma 
1 1 , info, rmax , 


0. 490203091522860 
12.0000000000000 


steplength, gamma 
tt, info, rmax. 


0. 895106789666395D-014 
1 . 00000000000000 



-633. 600054840564 
675. 884989041090 
•754.508656207635 
206.621797933154 

273601096408. 878 



1 . 00000000000000 

0 159079979.545477 



159079676. 124495 



158553666. 463820 



154534990. 248806 



154529135. 198641 



O 153934008. 178371 

1.00000000000000 7 

0 129555772.393379 

1 . 00000000000000 8 

0 6. 238418579101563D-0C 

1 . 00000000000000 9 

1 0. 596046447753906D-0C 



X 



r 0. 776390508831 104D-033 0. 433334237487128D-032 



0. 206074504049434D-031 
0. 204610797291700D-029 
0. 183015730011275D-027 
1 . 00000000000002 
-457763664.000000 
-457763664.000000 
-457763664. 000000 
-457763664. 000000 
-0. 23693557601 1567D-024 
-O. 293873582650006D-022 
-O. 279861690229253D-020 
-O. 242667168818140D-018 



0. 962964972193618D-031 
0. 917050802319426D-029 
0. 795171792462780D-027 
-457763664.000000 
-457763664.000000 
-457763664.000000 
-457763664. 000000 
O. 596046447753906D-007 
-O. 132243112192503D-023 
-O. 136357342349603D-021 
-O. 125185444231580D-019 
-0. 104770587172276D-017 



0. 446815747097839D-0 
0. 410207670714926D-0 
0. 343312265952184D-0 
-457763664.000000 : 

-457763664.000000 
-457763664. 000000 
-457763664.000000 
-0. 5960464477539060-00 
-O. 6288894668710120-02 
-0. 6244225884147320-02 
-0. 5585196742639730-01 
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f.’> 



RUN #9 



10 Newton steps and 



g rad lent step 



xO “834.972506153610 
-675.0 786^8947569 
“795. 176371801571 
-3a6. 762^6 7-4-5023a 



-983. 769175 140755 
924. 303418741492 
“0. 500703910308475 
322. 616508754233 



tt, info, rmax, 
1 1 , i I'l r o , rmax. 



7. 00000000000000 

15. 0000000000000 



aa7 . 07590103968. 
38. 94*4-2. r 74036471 
487. 15033467G203 
721 . 51 13*46799a89 
30022252 1 a28 . m-60 
17359103350370. 1 



steplength, gamma 
1 1 , info, rmax , 



54027. S046ly0032 
12. 0000000000000 



1 . 00000000000000 

0 1 7862 1 1 07 1 . 36 42 1 



step length, gamma 
1 1 , info, r ma x , 



15776. 5628664350 
la. 0000000000000 



0. 316619S73046375D-003 

0 1 7856a5 5 2 1 . a4 1 56 



steplength, gamma 
t t, info, rmax , 



27. 77192aa099574 
la. 0000000000000 



0. 250000000000000 

0 1339238345. 83143 



3 



steplength, gamma 
tt, info, rmax. 



8. 49827918067334 
12. 0000000000000 



1 . 00000000000000 

0 590430919. 804070 



steplength, gamma 
tt , info, rmax , 



2736. 52420367105 
13. 0000000000000 



0. 762939453125000D-005 

0 590426415. 173640 



steplength, gamma 
1 1 , i nf o , r max , 



47. 6924304507833 
14. 0000000000000 



0. 7ai250000000000D“002 

0 5358359S7. 15558^ 



steplength, gamma 
1 1 , info, rmax. 



8. 13635607249363 
15. 0000000000000 



0. 6250GOOOOOOOOOOD-001 

0 549466999. 076904 



steplength, gamma 
1 1 , info, rmax. 



a2. 5352574148447 
16. 0000000000000 



0. 328125000000000 

0 370527291.553912 



steplength, gamma 
1 1 , info, rmax , 



i . 55797420666023 
12. 0000000000000 



1 . 00000000000000 

0 133329412. 581193 



staple ngtfi, gamma 
1 1 , info, rmax, 



0. 504481 166507457 
12. 0000000000000 



1 . 00000000000000 



10 



0. 25033950805664 1 D“005 



steplength, gamma 0 . 93637729656047 1 E^-0 1 4 1 . 00000000000000 

tt, inf o, rmax, 12. 0000000000000 0 O. 000000000000000 

X , r 0. 165629975217302D-031 O . 597038282760043D- 03 1 



0. 1756448109281160-030 
0. 173795918181504D-029 
0. 1380506584136770-028 
1 . 00000000000000 
“457763664. 000000 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
-0. 505 462562 1 580 1 OD - 023 
“0. 1 26S34 1 7S633763D - 02 1 
“0. 10457 197565 0173D-020 
-0. 8185202122S340SD-020 



0. 422163843809682D-030 
0. 342661455705377D-029 
0. 26821 2707775 144D- 028 
“457763664. 000000 
-457763664. 000000 
-457763664. 000000 
“457763664. 000000 
0. 000000000000000 
-0. 162201621243004D-022 
“0. 257668357267525 Ei“02l 
-0. 2106a858a043524D“020 
-0. 1661 1 1454845750D-019 



0. 84432 76876 1 9364D “ 030 
0. 6902532920683e5D“029 
0. 5443 1 4024602a93D “ 023 
“457763664. 000000 
“457763664. 000000 
“457763664. 000000 
“457763664. 000000 
0. 000000000000000 
“0. 5360254147536100-022 
-0. 5303S30a 1 9667300-02 1 
-0. 4212971 68037048D ”020 



29 



RUN #10 



7 Newton steps 



xO 168. 562088607904 

-215.631795284871 
-312. 386754660958 
-783. 756813578536 



879. 936512051979 
386. 541205751119 
-847. 804830474609 
-462. 177283109723 



1 1 , info, r max , 

step length, gamma 
tt, info, rmax, 

steplength, gamma 
1 1 , info, rmax , 

steplength, gamma 
tt, info, rmax, 

steplength, gamma 
1 1 , info, rmax , 

steplength, gamma 
1 1 , info, rmax , 

steplength, gamma 
tt, info, rmax, 



18.0000000000000 

1935. 29079830120 
12. 0000000000000 

7019.68685306682 

14. 0000000000000 

78. 7484599706272 

13. 0000000000000 

335.697096172363 

14. 0000000000000 

3. 57321680138793 

12. 0000000000000 

0.490203091522860 
12. 0000000000000 



282.756764008947 

-272.694289835621 

-918.030969523965 

318.376253985250 

280160813516. 508 



steplength, gamma O . 9 1 3 1 77239355039D-0 1 4 
tt, info, rmax, 12. 0000000000000 



1 . 00000000000000 

O 851810879. 278383 

0. 122070312500000D-003 

0 851706898. 458158 

O. 781250000000000D-001 

0 785166074.844281 

O. 209960937500000D-001 

O 768681659. 969552 

1 . 00000000000000 

0 129555772.393379 

1 . 00000000000000 

0 O. 244379043579102D- 

1 . 00000000000000 

0 O . 000000000000000 



: , r O. 325482160601443D-031 
0. 200296714216273D 
O. 167632942359465D 
0. 134106353887572D 
1 . 00000000000000 
-457763664. 000000 
-457763664.000000 
-457763664. 000000 
-457763664. 000000 
-0. 993292709357019D 
-0. 125072596775842D 
-O. 101562710163842D 
-0. 806483150338064D 



O. 986076 131 526265D-031 



030 


0 


029 


0 


028 


0 




0 


023 


-0 


021 


-0 


020 


-0 


020 


-0 



. 264268403249039D 
457763664.000000 
457763664.000000 
457763664.000000 
457763664. 000000 



■030 O. 819675784331207D-0 
■029 O. 680392530753123D-0 
■028 O. 528536806498078D-0 
-457763664.000000 
-457763664.000000 
-457763664. 000000 
-457763664. 000000 
O. 000000000000000 



51 1575 1326771 30D-0 
O. 409260106141704D-0 



022 


-0 


021 


-0 


020 


-0 


019 
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CD O 



RUN #11 



10 f^l0wton st^ps ^nd Z j^rc<di0nt st»5p 



xO -507. 462338260756 
460. 05337501S011 
“554. 3a56381 16258 
-905. 916211360387 



-745. 331526294334 
-77. 6a30245183148 
-506. 427349605299 
-924. 250809940748 



1 1 . info, r ma x , 
1 1 , Info, r m a X , 



9. 00000000000000 

14. 0000000000000 



822. 626989288406 
-406. 981993993563 
53. 2199324551891 
901. 741966752070 
282058958142. 296 
9678403149694. 32 ’ 



steplength, gamma 
1 1 , i n f o , r m a X , 



45364. 5042313625 
0000000000000 



1 . 00000000000000 

0 4,i.9 1 07069 . 64 1 468 



1 



s t e p 1 e n g t h , gamma 
1 1 , info, rma X , 



15773. 5499860498 
14. 0000000000000 



0. 74863433837S906D-004 

0 429074945.212750 



step 1 ength, gamma 
1 1 , info, r m a X , 



29. 2671518584511 

14. 0000000000000 



0. 625000000000000D-001 

0 4022 58Sa 8 . 2 73 4 1 6 



s t e 1 e n g t h , gamma 
1 1 , info, rma x , 



10. 387500633555^ 

18. 0000000000000 



0. 375000000000000 

0 295236784. 148774 



staple ngth, gamma 
t t , info, rmax , 



2. 41364909823324 
13. 0000000000000 



1 . 00000000000000 

0 287724616. 031283 



s tap 1 en g t h , gamma 
1 1 , info, rmax , 



106. 889616604460 

16. 0000000000000 



0. 244140625000000D-003 

0 287655027. 217895 



6 



step! ength, gamma 
1 1 , info, rmax. 



9. 48881403921367 
13. 0000000000000 



0. 156250000000000D-001 

0 233221899.6635^3 



staple ngth, gamma 
1 1 , info, rmax , 



1 1 . 5704315204349 

19. 0000000000000 



0. 156250000000000D-001 

0 278861295.027645 



step length, gamma 
1 1 , info, rmax. 



0. 911937664675868 
12. 0000000000000 



1 . 00000000000000 

0 135022355. 068402 



step length, gamma 
1 1 , info, r ma x , 



0. 510872132671297 
12. 0000000000000 



1 . 00000000000000 



10 



0. 2503395080566alD-005 



stepl ength, gamma 0. 935 6286 4928736 4D-0 1 4 1 . 00000000000000 

1 1 , i n f o , r max , 12. 0000000000000 0 0 . 000000000000000 

X , r 0. 93889084788S777D-032 0 . 337037740267766D-03 1 



0. 1 0 0 1 4 335 7 1 08 1 36 D - 030 
0. 156539585879795D-029 
0. 1 3S0506584 1 3677D -028 

1 . 00000000000000 
- 457 76366 '-i- . 000000 
-45776366a. 000000 
-4b77b36o4. U 00000 
-457763664. 000000 
-0. 23S5z-6743083756D -023 
-0. 8040381221304 16D- 022 
-0. l0632892a764930D-020 
-0. 8185202l22S3a06D-020 



0. 2634672163 9 2174D-030 
0. 350057G26691824D-029 
0. 268212707775144D-028 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
0. 000000000000000 
-0. 102855753 9 27502 D -022 
-0. 20 1 2a462339S724D-02 1 
-0. 213657349529860D-020 
-O. 1 63704042456682D-0 1 9 



0. 659438412958189D-030 
0. 700114G53383648D-029 
0. 536425415550286D-028 
-457763664. 000000 
-457763664. 000000 
-457763664, 000000 
-457763664. 000000 
0. 000000000000000 
-0. 30562S525956006D-022 
-0. 4 7 7 7 2089 5 9 5 5649 D — 02 1 
-0. 421297 168Go7048D-020 
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RUN #12 



10 Newton steps 



xO “415. 465009220958 
“777. 129532256195 
-976. 113784378833 
“78. 1654829784306 



93. 3967863568357 
503.520149216688 
686. 576335880251 
169.694047289757 



tt, info, rmax, 

steplength, gamma 
1 1 , info, rmax , 

steplength, gamma 
tt, info, rmax, 

steplength, gamma 
tt, info, rmax, 

steplength, gamma 
1 1 , info, rmax , 

steplength, gamma 
tt , info, rmax , 

steplength, gamma 
1 1 , info, rmax , 

steplength, gamma 
1 1 , info, rmax , 



19. 0000000000000 

2075. 58548391905 

12. 0000000000000 

4. 19354626531279 

13. 0000000000000 

581. 479889666029 

14. 0000000000000 

17. 0801205817469 

17.0000000000000 

6. 52560684331984 

15.0000000000000 

2. 39912596957116 

15.0000000000000 

18. 3964281018146 

19. 0000000000000 



“655.692275827660 
“632. 224189726941 
263. 850657800655 
“953. 766508610371 

669201674021. 751 



steplength, gamma 0.662869275097632 
tt, info, rmax, 12. 0000000000000 

steplength, gamma 0. 510872132671297 
tt, info, rmax, 12.0000000000000 



1 . 00000000000000 1 

O 495718564. 127140 

0.500000000000000 2 

0 356977880. 225168 

0. 610351562500000D-004 3 

O 356957986. 239844 

0. 781250000000000D“001 4 

O 331543988. 701578 

0. 500000000000000 5 

0 181610652. 924412 

0. 125000000000000 6 

0 162997667. 387076 

0. 218750000000000 7 

0 134566511 . 091070 

1 . 00000000000000 8 

O 135022355. 068401 

1 . 00000000000000 9 

0 0. 250339508056641D“00 



■014 



steplength, gamma 0 . 938228545 1 89278D 

tt, info, rmax, 12.0000000000000 

X , r 0. 938890847888777D“032 0.3428155301 



10 



0. 100918729085891D“030 
0. 156539585879795D“029 
O. 138050658413677D“028 
1 . 00000000000000 
“457763664.000000 
“457763664.000000 
“457763664. 000000 
“457763664. 000000 
-O. 286526743083756D“023 
“0. 818144054097616D“022 
“0. 106828924764930D“020 
“O. 830557274228752D“020 
PAUSE? 



0.2680894482587030“ 
0. 3500570266918240“ 
0. 2721570123012490“ 
“457763664.000000 
“457763664. 000000 
“457763664. 000000 ‘ 

-457763664. 000000 
0. 000000000000000 
“0. 1046189954234020“ 
“0. 2012446293987240“ 
“0. 2136578495298600“ 
“0. 1661114548457500“ 



00000000000000 
0 0. 000000000000000 
00928D-031 
030 0. 6594384 12958 189D-03 

029 0. 7001 140533836480“ 02 

028 0. 5443140246024980-02 

“457763664. 000000 
“457763664.000000 
“457763664. 000000 
“457763664. 000000 
0.000000000000000 

30797951 46 17206D-0: 
477720895955849D-0: 



022 


“0 


021 


“0 


020 


“0 


019 
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RUN #13 



6 Newton steps 



2 ^fcidient 



t e p 5 



xO 



tt , 
1 1 , 

s t e 
tt, 

s t e 
tt, 

3 t e 
tt, 

s te 
tt , 

s t e 
tt, 

s t e 
tt, 

s t e 
tt, 

X 



536. 477440169505 
-565 . 2^^3557 1 1304 
-SO. 3906713861346 
157. 333736486^45 



961 . 442495391034 
ISO. 13253650S556 
-628. 92944290951 1 
42. 1 332 7 4 5 4 28034 



449. 092869887492 
768. 913749734951 
273 . 2492742377 1 4 
528. B7 1667306834 



into, £ ma x , 
i ri i o , r iTi a X , 



4. 00000000000000 

15. 0000000000000 



1 191934030739. IbB 

1 16845691099344. 1 



plength, gamma 
info, r m a X , 



371 17. 6105912337 
14. 0000000000000 



1 . 00000000000000 

0 939465467.781931 



pi ength, gamma 
info, rmax , 



1 1 . 6912012042131 
14. 0000000000000 



0. 500000000000000 

0 532 1 53589 . 895607 



plength, gamma 
info, rmax. 



2l . 0876 40d 66 0046 
15. 0000000000000 



0. 500000000000000 

0 233305905.447369 



3 



plength, gamma 
info, r ma x , 



75. 6961499181736 
16. 0000000000000 



0. 226562500000000 

0 219586876.653159 



plength, gamma 
info, rma x , 



1 . 44623443931224 
12. 0000000000000 



1 . 00000000000000 

0 133329412. 581 193 



plengtfi, gamma 
info, rmax. 



0. 504481166507457 
12. 0000000000000 



1 . 00000000000000 



0. 2503395080566410-005 



plength, gamma 
info, 1 ma x , 



0. 93425201 1646448D-0 14 
12. 0000000000000 



1 . OOOOOOOOOOOOGO 

0 0. 000000000000000 



, r 0. 1617781 1532S528D-031 0 . 597038282760043D-03 1 



0. 174104066972606D-030 
0. 173795918181504D-029 
0. 136073506150625D-028 
1 . 00000000000000 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
0. 493707618852010D-023 
0. 128834178633763D-021 
0. 104571975650178D-020 
0 . SI 85 2021 2263408D -020 



0. 422163343S09682D-030 
0. 34266 1 455705377D- 029 
0. 268212707775 144D-028 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
0. 000000000000000 
-0. 1 8220 1 6 2 1 243004D - 02 2 
-0. 257668357267525D-021 
-0. 2 1 06485840435240 -020 
-0. 1 6 1 29663006 76 1 3D -0 1 9 



0. 8443276376193640-030 
0.6902532920683850-029 
0. 5285363064930730-028 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
0. 000000000000000 
-0. 5313234374312100-022 
-0. 5303830419667300-021 
-0. 4152736371143760-020 
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RUN #14 



6 Newton steps 3 gradient steps 



xO 



-313. 188Z77293696 
225. 308342001 151 



749. 187281293977 
21 . 6374771325915 



-74. 7127697490351 
-449. 303067641651 
334. 306731674827 
834. 136420883962 



363. 859180360o79 
-722. 721760127721 
566. 136306586745 
-47 1 . 49331 15G3622 



tt, inf o, r max, 8. 00000000000000 

1 1 , i nf o , X max , 1 . 00000000000000 

1 1 , i n f o , r ma X , 17. 0000000000000 



1 220557173979.063 ^ 
1 2819394490440.15 ^ 
1 743317395805433. * 



step length, gamma 895425. 145414435 
t t , i n f o , r ma X , 13.0000000000000 



1 . 00000000000000 1 
0 153587264.635352 



staple ngth, gamma 
1 1 , info, rmax , 

staple ngth, gamma 
t t , info, rmax , 

staple ngth, gamma 
1 1 , info, rmax, 

stej:» length, gamma 
tt, into, rmax, 

staple ngth, gamma 
tt, info, rmax, 

steplength, gamma 
1 1 , info, rmax. 



98. 77S4435622585 

15.0000000000000 

13. 9259737920949 

14.0000000000000 

16. 3951704458082 

15. 0000000000000 

0.7171 12636602673 

12. 0000000000000 

0. 499630146021219 
12. 0000000000000 

0. 91542631 17366 77D-0 14 
1 . 00000000000000 



0. 732421875000000D-002 2 

0 153108097. 868560 

0. 1 17137500000000D-001 3 

0 152485904. 282359 

0. 625000000000000D-001 4 

0 150818862. 186507 

1 . 00000000000000 5 

0 132047263. 790020 

1 . 00000000000000 6 

0 0. 244379043579102D-0 

1 . 00000000000000 

1 0. 596046447753906D-0 



X , r 0.77037197775481 
0. 208000433993821D-031 
0. 2021^56069628840-029 
0. 1830157300112750-027 
1 . 00000000000002 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
-0. 235098866120005D-024 
-0. 293873582650006D-022 
-0. 279861690229253D-020 
— 0. 242667 163S1S140D — 018 



D-033 0. -^3092682505664 

0. 96296497219361SD-031 
0. 917050802319426D-029 
0. 795171792462780D-027 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
0. 596046447753906D-007 
-0. 1 31 508428235878D-023 
-0. 1 344765 5 1 420643D - 02 1 
-0. 1251654442315S0D-019 
-0. 103229843243272D-017 



4D-032 i 

0. 440652771275800D-0| 
0. 410207670714926D-0 
0. 338263556158770D-0 
-457763664. 000000 
-457763664.000000 
-457763664. 000000 
-457763664. 000000 
-0. 596046447753906D-0 
-0. 6347669385240 1 2D-0 
-0. 616899424698392D-0 
"0. 558519674263973D-0 
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RUN #15 



8 Newton steps 



xO 296.981125772494 
-40.5170014433711 
78. 1184786292034 
95. 5765248137808 



805. 619158814028 
-670.077290135360 
-737. 687608386972 
-613. 386207147848 



51 . 6047080018680 
-975. 512313377759 
-6.90543145820155 
-628. 146325780695 



1 1 ♦ into, rmax , 


19. 0000000000000 


0 744985996579.413 




step length, gamma 
t t , info, rmax , 


1863. 45202256309 
14.0000000000000 


1 . 00000000000000 

0 151289322.544588 


1 


steplength, gamma 
tt , info, rmax , 


20. 6857895261084 
15. 0000000000000 


0. 253906250000000D-001 

0 150658392.083229 




steplength, gamma 
tt, info, rmax. 


2. 08038227471001 
16. 0000000000000 


0. 250000000000000 

0 144578667.929296 


3 


steplength, gamma 
t t , info, rmax , 


35. 9811215440998 
18. 0000000000000 


0. 488281250000000D-003 

0 144551342.269987 


4 


steplength, gamma 
tt , info, rmax , 


6.59190252887505 
19. 0000000000000 


0. 781250000000000D-002 

0 144407436.410621 


5 


steplength, gamma 
t t , Info, rmax , 


0.455879229119055 
12. 0000000000000 


1 . 00000000000000 

0 135022355.068402 


6 


steplength, gamma 
tt , info, rmax, 


0.510872132671297 

12.0000000000000 


1 . 00000000000000 

0 0. 250339508056641D 


7 

-00 


steplength, gamma 
tt , info, rmax , 


0. 938331531860259D-014 
12. 0000000000000 


1 . 00000000000000 
0 0.000000000000000 


8 



X , r 0. 938890847888777D 
0. 100918729085891D-030 
0. 156539585879795D-029 
0. 140022810676730D-028 
1 . 00000000000000 
-457763664.000000 
-457763664.000000 
-457763664.000000 
-457763664. 000000 
-0. 286526743083756D-023 
-0. 818144054097616D-022 
-0. 106828924764930D-020 
-0. 830557274228752D-020 



032 0.342815530 

O. 268089448258703D 
0. 350057026691824D 
0. 2721 5701 230 1249D 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
-457763664. 000000 
0.000000000000000 
O. 104618995423402D 
0. 205006211256644D 
O. 213657849529860D 
0. 1661 1 1454845750D 



100928D-031 



-030 

-029 

-028 



022 


-0 


021 


-0 


020 


-0 


019 





O. 671764364602268D-03 
0. 7001 14053383648D-02 
0. 544314024602498D-02 
-457763664. 000000 
-457763664. 000000 
-457763664.000000 
-457763664. 000000 
0.000000000000000 
0. 3079795 1461 7206D-0 
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